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ABSTRACT 

Sensory systems tcike continuously varying stimuli as their input 
and encode features relevant for the organism's survival into a se- 
quence of action potentials - spike trains. The full dynamic range 
of complex dynamical inputs has to be compressed into a set of 
discrete spike times and the question, facing any sensory system, 
eirises: which features of the stimulus are thereby encoded and how 
does the animal decode them to recover its external sensory world? 

Here we study this issue for the two motion-sensitive HI neurons 
of the fly's optical system, which are sensitive to horizontal velocity 
stimuli, each neuron responding to oppositely pointing preferred 
directions. They constitute an efficient detector for rotations of the 
fly's body about a vertical etxis. Surprisingly the spike trains psit) 
generated by an empoverished stimulus (t), containing just the 
instants when the of velocity S{t) reverses its direction, convey the 
same amount of global (Shannon) information as spike trains p{t) 
generated by the complete stimulus S{t). This amount of informa- 
tion is just enough to encode the instants of velocity reversal. Yet 
this suffices to give the motor system just one, yet vital order: go 
left or right, turning the HI neurons into efficient analog-to-digital 
converters. Furthermore also probability distributions computed 
from p{t) and ps(t) aire identical. Still there are regions in the 
spike trains following velocity reversals, 80 msec long and contain- 
ing about 3-6 msec long spike intervals, where detailed stimulus 
properties are encoded. We suggest a decoding scheme - how to 



reconstruct the stimulus from the spike train, which is fast and 
works in real time. 
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Introduction 

All living organisms rely on the sensory nervous system to quickly and re- 
liably extract relevant information about a changing external environment. 
E.g. the visual system receives time-continuous optical flow patterns and 
generates a set of discrete identical pulses, called action potentials or spikes. 
This analog-to-digital conversion means that only a small fraction of rele- 
vant stimulus properties is actually encoded in the spike train and a lot of 
effort has been expended in finding out just which and how [20l |35l [151 EH 
[T6| [T8| [T71 |32| l27t [30| [H]. Debates have been going on for decades over 
the way and means employed by the organism to effect this dimensional 
reduction^^ 1221 ESI E21 [221 [1] without losing essential information. Al- 
though these studies have revealed a wealth of interesting properties, a much 
more direct approach would be to investigate the animal's performance to a 
stimulus, manufactured to contain only relevant features. Once these have 
been successfully guessed and appropriately tested, we need a decoding algo- 
rithm to reconstruct the stimulus from the spike train. It should work in real 
time to be available to the animal, using only data which are presumably 
held in a memory. 

To explore these issues we recorded spikes from the two motion sensitive 
HI neurons - one for each compound eye - of the fly Chrysomya megacephala. 
Each HI neuron is sensitive to horizontally moving stimuli, being excited by 
back-to-front motion and inhibited by the oppositely moving scenery. They 
measure rotational velocities of the fly's body around a vertical axis [12] . In 
our experiments the fly sees a rigid pattern moving with horizontal velocity 
S{t) either on a Tektronix monitor or on a large translucent screen - see Meth- 
ods for details. From S{t) we manufacture an empoverished version ^^(t), 
which tells the fly only which way the scenery is rotating: either to the left 
or to the right, ^^(t) generates spike trains psit), which are indistiguishable 
from the ones p(t) generated by S(t), as far as global averages over the whole 
experiment are concerned. Yet we discover specific local regions on a time 
scale of 50 — 100 msec, where the responses p(t) and psit) do differ. Thus 
the encoding and decoding process should occur in a multilayered fashion, 
involving several time scales. 

Generating the same response raster with boxed stimuli 

Consider one HI neuron and let its excitatory stimuli be positive {S{t) > 0) 
and inhibitory negative {S{t) < 0). Instants when the stimulus velocity 
reverses sign correspond to the zero-crossings of the stimulus S{t). We now 
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discretize our continuous stimulus S(t) and generate a discrete boxed version 
Ssit), assigning different values to positive and negative velocities: 




So, \fS{t) > 
-So,\fSit) < 0. 



The constant 5*0 will be chosen so that S{t) and 5*5 (t) have the same 
variance We emphasize that S{t) and 5*5 (t) have the same zero-crossings. 

We subject the ffy alternately to stimuli S{t) and ^^(t), each being 10 
sec long. They are repeatedly shown the ffy, the odd/even repetitions being 
due to S(t)/SB(t) respectively. From the responses of the HI neuron we 
construct a raster, each dot representing a single spike. In Figjpi we show 
a section of this raster plot with 35 repetitions for each stimulus, where we 
collected the responses due to S(t) below repetition 35 and the ones due 
to SBit) above. The result is completely unexpected: visually there is no 
significant difference between repetitions. Notice that the plots of the screen 
positions (f){t) and 0b (t) emphasize, that the ffy does not even view the same 
picture, when subjected to S{t) and Ssit). Apparently the ffy's HI sensory 
system does not pay attention to any details of stimulus S(t), which are lost 
in Ssit). 

To further explore this striking fact, we generate a whole set of stim- 
uli S'Ar(t) adding a certain amount of random noise to but always 
preserving the zero-crossing d The noise added is measured by the ratio 
/ = a{S — Siy)/a{SN), where a is the stimulus variance. Subjecting the ffy 
to the set SB(t), SN{t), we obtain the raster of FigjljD. Repetitions 1 : 35 
are due to Ssit) and the the remaining ones due to S]\f(t), the noise being 
different for each repetition 36 : 70. Again no difference can be seen between 
the two sets. For a quantitative test, we compute the interval histograms 
due to pAr(t) and pBif), shown in Fig|2] the are the same. Additionally all 
the rank-ordered word distributions of pAr(t) and psit) look identical. In 
the insets of Figj2] we plot these distributions for binary words of lengths 
L = 5, 8. This astonishing identity also hold for longer words, although the 
word lables for L > 7 are then not all identical. 

This equivalence holds true for a host of global statistical averages. Ex- 
amples, which have been used a lot to characterize neural responses, are 
the Shannon mutual informations (MI) /[pjS"], I[pb\Sb], which the spike 

'At instants ti, where the addition of noise would have changed the sign of S{ti), we 
replace SN(ti) by — S'Ar(ii). 
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Figure 1: Rasterplots of spikes generated by S{t) and its boxed and noisy 
versions, a: Section of rasterplots for spikes generated by S{t) and SB{t). S{t) 
is a gaussian velocity signal with correlation time r = 60 msec, the raster-time 
running from 7200 to 8200 msec Repetitions 1 : 35 are due to S{t), whereas 36 : 70 
are due its boxed versions SB{t)- The bottom graph shows the actual screen 
positions (/>(t) and (pB{t) (deg) as seen by the fly for S{t) and Ssit). Experiment: 
Tektronix, r = 60 msec, b: Section of rasterplots for spikes generated by Ssit) 
and noise-added stimuli Sj\fit)- Repetitions 1 : 35 are due to Ssit), whereas 36 : 70 
are due noise-added gaussian stimuli S^it)- On top we show the superposition of 
all stimuli S'Ar(t). Experiment: Natural, r = 200 msec, / = 0.5. 
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Figure 2: Interval histograms for noise added stimuli. Compare the interval 
histograms generated by stimuli S^it) and SB(t). The black (red) curve is the 
interval histogram of /07v(t) (pBit))- Upper inset: Five letter word distributions 
for pN^t) and psit)- The time axis was discretized into 2 msec wide bins. The 
presence of a spike in each bin was represented by 1 and the absence by 0. Each 
binary word is a sequence of five I's or O's and we have 2^ = 32 possible word labels. 
We plot the rank-ordered word distribution for pAr(t) and psit), using identical 
labels for both histograms. Their difference, upscaled by 10, is shown as the dashed 
(blue) curve. Lower inset: Same as upper inset for the first thirty eight letter 
words, but now the word labels W{i) and Wsii) for pN{t) and pB{t) are not all 
identical. E. g. W{j) = WBij),j = 1,2,3,4, but tF(5) = TFb(6),VF(6) = VFb(5). 
Experiment: Tektronix, r = 60 msec, / = 0.5. 
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trains p(t), psif) convey about the stimuli 5'(t), Ssif) respectively. Us- 
ing the direct method of Ref. [28], we plot in FigjS^ the total entropies 
E[p\,E[pb\ of the spike trains and the entropies conditioned on their stim- 
uli E[p\S],E[pb\Sb\- Informations are given by /[Sip] = E[p\ — E[p\S], 
I[Sb\pb\ = E[pb\ — E[pb\Sb\- Notice that p{t) conveys the same amount of 
MI about the total stimulus S{t) as psit) conveys about the boxed stimulus 
Ssif)- Thus, as far as the MI is concerned, p{t) ignores all the features in 
S{t), which are absent in Ssit), paying attention only to whether the scenery 
moves left or right. Since the entropy of SB{t) is about ten times smaller 
than S'(t)'s, this amounts to a huge entropy reduction with a concomitant 
increase in coding efficency e, defined as the ratio of MI divided by stimulus 
entropy. 

The boxed stimulus Ssit) is characterized by its zero-crossings, half of 
them being from negative to positive stimulus values (upcrossings) and the 
other half the opposite. The information to locate all the upcrossings lupzc 
is about equal to /[pl^], as shown in FigjS^. It is therefore consistent to 
assume that one HI neuron extracts enough information to locate the up- 
zero-crossings. Recording from both HI neuronsp], we obtain an efficiency 
e = 0.42, again just enough to locate all (up and down) zero-crossing^ of 
the stimulus E3 

Two coding regions after zero-crossings 

Up to now we have endeavored to show that the HI neuron reponds equally 
to S{t) and its boxed version, but there could still be subtle differences, if we 
look more closely. To unravel differences between p{t) and psit) we select 
]:>romment zero-crossings (ZC). These are instants, when the velocities S{t), 
Ssit) change from negative to positive values leading to well defined onsets 
of spiking activity- see Methods for details. We compute spike rates after 
each ZC for a time interval of 400 msec, shown in Fig|4| We observe that 

" Under some technical assumptions signals can be reconstructed from their zero- 
crossings (111 131] ■ 

Since our boxed stimulus 5*5 (i) has low entropy, it is actually possible to compute the 
MI as I[Sb\pb] = E[Sb] — E[Sb\pb]- This yields the same value as the direct method [28] 
and we also check the symmetry = I[pb\Sb]- As additional bonus we notice that 

undersampling problems can be tamed, since we explicitly know the entropy E[Sb] for all 
word-lengths. Values for this entropy obtained from our data can therefore be corrected. 
If we assume that the undersampling problems for E[Sb] and E[Sb\pb] are identical, the 
same correction can be applied to the MI I[Sb\pb]- 
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Figure 3: Information and entropy rates, a: Information and Entropy rates 
vs. 1/wordlength. Spike times were discretized at 2 msec and the cell's responses 
were represented as binary words of length L bins. The word-entropies E = 
E[p\ and Eb = E[pb\ are identical. = E[p\S] and Nb = E[pb\Sb] - the 
entropies, which the distributions of words convey about the respective stimuli - 
and / = /[pis'], Ib = I[pb\Sb\ coincide within errors. Errors are ~ 10%. lupzc 
is the information-rate to locate up-zero-crossings. Experiment: Tektronix, r = 
60 msec. b: Delay times from I[Sb-,p\, I[Sb,Pb] and raster plots. Squares, 
circles are delay times of common ZC's for p, pb extracted directly from raster 
plots. To compute I[Sb,p] and I[Sb,Pb] time is binned into 2 msec. To obtain 
the required probability distributions pr[SB\pB]itd)^ P'f'iSBlplitd) we proceed as 
follows. We sample the spike-train distribution pB{t) through a window four bins 
long, obtaining a set of binary words Wb with their multplicity niB and their 
occurrences ti. For each word Wb we sample ms stimulus words Wsg of size four, 
but at instants ti—td- Varying td we compute pr[Wsg\WB]{td) to obtain I[Sb,Pb]- 
We repeat the procedure for p{t). Experiment: Tektronix, r = 160 msec. 
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Table 1: Mean variance of first and second spike after all ZC's for stimuli 5* 
and Sb vs. correlation lengths : Mean variance of 1** spike for 

p{t) {pB{t)); a^'^^ i^B^)'- Mean variance of 2""^ spike for p{t) {pB{t)). For each 
stimulus S we find all ZC's and compute the variances a of the 1'^* and 2"'^ 
spike after each ZC. The mean a is taken over the three types of experiments 
(Natural, Bars, Tektronix), using a total of N'^{N]^) zero-crossings for p{t) 
{pB(t)) respectively. 

there are in fact two coding regions I and II, following the first spike after 
ZC's. Region I spans a time Ti ~ 80 msec. Here the rate for pB(t) is larger 
than the pb(^)'s, reflecting the stronger response to sharper onset of positive 
stimuli. This is followed by region II, where the rates for p{t) and pB{t) are 
the same. 

To take a closer look at regions I and II, we compute in each region in- 
tervals common to p{t) and PBit), accumulating data (~ 50.000 data points) 
from ZC's of all experiments. The histograms of these intervals are shown in 
the inset of Figjl} Both histograms peak around 6 msec, but their shape is 
different. This could encode the " YES, adjust to turns " information, which 
the HI neurons should extract from the stimulus and relay to the motor 
system. Since the variances of S(t) and SB(t) are the same, the spike trains 
should encode this and presumably other ensemble properties in Region II |B]- 
A more detailed investigation of regions I and II could allow the extraction 
of the precise interval structure associated with the turning command. Due 
to the - on average - sharper onset of the boxed stimulus SB{t) after ZC, the 
spiking precision of the first and second spike after all ZC's is always better 
for pb as shown in Table [TJ This underscores the sensibility of the HI neuron 
to discontinuities in the stimulus. 

A further boxed vs. complete stimulus dependent property are the delays 
- the times it takes the fly to generate the first spike after ZC. These times 
exhibit a pronounced dependence on the stimulus history as shown Fig|3]3. 
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Figure 4: Spike rates after ZC. Sum of spike rates of p{t) (black) and psit) 
(red) summed over 71 ZC's. Tlie time axis is discretized into 2 msec bins and 
translated so that the first spike after ZC occurs at t = 0. /J_B(i)'s spike rate is 
always larger and merges with /o(i)'s after ~ 80 msec. Experiment: Tektronix, 
r = 400 msec, / = 0.5. Inset: Histograms for intervals (< 30 msec) common to 
p{t) and PB{t) for regions I and II, using all experiments. 
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Here we plot delay times, extracted directly from raster plots, for p{t) as 
circles and for psit) as squares. For comparision we also compute mean 
delay times td extracted from the Mi's ( I[Sb\p], I[Sb\pb] ), which the spike 
trains p{ti) and PB(ti) convey about the boxed stimulus ^^(t). When plotted 
as a function of they show a pronounced peak, which provides a mean- 
information- delay-time. As we move off the peaks, the correlation between 
stimulus and spike-train vanishes and - although not shown in the figure - all 
information rates actually do vanish for large \td\. Notice that the peak values 
are identical to the MI obtained by the direct method of Ref. [28]. In all cases 
the neuron responds faster by ~ 10 msec to the boxed stimuli, emphasizing 
again the systems sensitivity to sharp stimulus variations, newline 
Sensitivity of HI to temporal discontinuities 

If one of the foremost aims of the HI neurons is the extraction of temporal 
discontinuities from the optical flow, in order to respond preferentially only 
to these, the fly has to decide on a threshold: how much stimuli have to rise 
above the background for them to be classified as discontinuities? 

Let us measure discontinuities by changes in stimulus variance 6a, defin- 
ing / = 6a /a. To study this situation, often stimuli are chosen, which are 
piecewise constant. Since the variance of a constant equals zero, / = oo 
and the fly obviously interprets this as a discontinuity, even if 6a is small. 
To simulate a more realistic situation, we expose the fly to a series of ran- 
dom gaussian stimuli S'„(t) with correlation times r = 200 msec and vari- 
ances an- They are generated from a common stimulus So{t) with stan- 
dard deviation ao = 7 deg, by upscaling its standard deviation as (T„ = 
n * cro/100,n = 1,2,3, ..,45. FigjS^ shows the spike rates generated by the 
stimuli on top. The zero-crossing for n = 1 occurs at ti = 475 msec and for 
n = 45 at ^45 = 478 msec with intermediate values for the other n-values. 
These zero-crossing initiate the large spike-rate at t ~ 510 msec. To expose 
the structure of this peak's spike patterns, we compute the distributions for 
all 3 letter words W^^^\n),n = 1, 2, 3, .., 45, i = 1,2, ..,8 = 2^ in the peak, 
shown in FigjSjD. The distributions change for up jump sizes n > 3. To em- 
phasize this, we subtract the distributions for the smallest up jump to get 

dwi^\n) = W}^\n) - Hi) and take their mean dwf ^ =< dW,^^\n) >„. 
In FigjS}: we show these means for word-sizes 1, 2, 3, 4, 7, 8, which clearly show 
a steep rise starting at the 3% upjump. A distributional encoding of the vari- 
ance can be seen: words of all lengths contribute to small (n < 15) upjumps, 
whereas large upjumps are encoded into words of length < 4. Thus under 
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Spike rates and word distributions after upjumps 
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Figure 5: a. Spike rates (poststimulus time histograms) after upjump for jump- 
sizes n * cr/100, n = 1, 2, ..,45. The peak between 500 and 600 msec is due to the 
zero-crossings at f ~ 475 msec - see stimuH on top. Its structure clearly depends 

on stimulus variance, as revealed by the two plots below, b: 3 letter word distribu- 

(3) 

tions VF- (n) in the peak plotted against n. The word distribution for no spikes 
([000]) is downscaled by a factor of 100 and the one for all occupied by spikes 
([111]) is upscaled by a factor of 10. Binsize is 2 msec, c: Mean of the word- 
distribution difference Wi{n) — Wi{l) for word-lengths 1,2,3,4,7,8. Experiment: 
Tektronix, r = 200 msec. 
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our experimental conditions the threshold for detection is a ~ 3% change in 
variance. 

Discussion 

An efficient representation of the sensory input is one of the main require- 
ments to be satisfied by a neural codef4]. Shannon's mutual information 
between stimulus and spike train is often used to asses this efficiency. Our 
results show that this measure is too coarse, since it is unable to distinguish 
the complete stimulus from its boxed version. We find in fact that the mutual 
information is only sufficient to encode the zero-crossings of the stimulus. Yet 
a closer look reveals a multilayered scheme [51 [3] of which we observe two lay- 
ers, suggesting the following scenario. In a region 80 msec long after spiking 
onset following zero-crossings we observe spike patterns, about 6 msec long, 
which encode turning commands, whereas stimulus ensemble properties, like 
stimulus variance, are encoded on time scales longer than 80 msec. If vari- 
ance changes together with zero-crossings, the fly's detection time is much 
shorter: from the data of FigjS] we extract a time-scale of ~ 37 msec for a 
change in stimulus variance of > 3%. 

Once we know that the spike train carries just enough information to 
encode the position of all the zero-crossings, what does this tell us about the 
decoding problem: how does the fly recover the stimulus from Hi's 
spike train? This information allows the extraction of the zero-crossings 
(or reconstruction of the boxed stimulus Ssit) up to a scale factor) rather 
straightforewardly for stimulus correlation times r > 200 msec: select a 
spike train for some repetition in a raster and search it for 10 msec long 
gaps, followed by 3-4 spikes less than ~ 10 msec apart - this will locate 
up-zero-crossings ZCsp They coincide with all the up-zero-crossings ZC^ 
obtained from the spike rate. Peaks signalling onset of spiking activity and 
locating ZCr, are defined as spike rates larger the 60% of the repetition 
number and no preceeding spiking activity for 20 msec. This definition of 
ZCr excludes zero-crossings too small to be relevant for the fly and extracts 
only the important ones. An identical procedure applied to the responses of 
the contra-lateral HI neuron locates the down- zero-crossings. For correlation 
times smaller than 200 msec more elaborate schemes have to be adopted. 

Since not all zero-crossings are equivalent it stands to reason, that more 
information is allocated to the important ones. It is thus probable, using 

''^ This recipe is only a rough guide to illustrate a possible path to extract ZC's. 
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the spike patterns stored in region I, that more than SB{t) can be recov- 
ered from a spike train. Since we know that the HI neuron has acces to 
the mean and variance of the stimulusfS], a fast local reconstruction proce- 
dure could be accomplished in two (or more) stages. First the important 
zero- crossings would be extracted from the spike trains, the other encoded 
properties supplying the scale factor required to reconstruct SB(t). In the 
second stage more stimulus details could be included using e. g. the usual 
Volterra series reconstruction |22l |T0], although this requires the knowledge 
of stimulus-spike correlation functions like < S(ti)p(t2) >, whose update at 
the milisecond time scale is problematic. 

We have only analyzed the output of HI neurons and suggested a way 
to extract the specific sequence of spikes, which would relay the turning 
command to the motor neurons. The fly uses also mechanosensory organs, 
gyroscopes using Coriolis forces, to detect fast self-rotations |2ll E]. These 
are the halteres, beating at the same frequency as the wings. The visual and 
mechanosensory inputs are then fused to obtain a more robust estimate of 
the stimulus ITT] . Therefore we have to keep in mind, that there are many 
other sensory pathways, which respond to stimulus details not detected by 
the HI neurons. 

Methods 

Experimental Setup and Preparation 

Immobilized flies were shown a rigidly moving scenery, while action poten- 
tials - spikes - were recorded extracellularly from HI neurons. We used two 
experimental setups: Tektronix and Slide. In the Tektronix setup, the fly 
views vertical bar patterns on a Tektronix 608 monitor, whose update-rate 
was 500 Hz, so that the bar pattern moved by 6x = v{t) every 2 ms[Tl [2]. In 
the Slide setup the fly views a large ( 60 cm x 40 cm ) translucent convex 
cylindrical screen, a slide being projected on the concave side via a simple 
optical system containing two mirrors. These are analogically controlled by 
linear motors, cannibalized from hard disk controllers. They move the slide's 
projection horizontally across the screen[7]. This setup produces a conti- 
nously moving image, but its performance is limited by mechanical inertia 
and fatigue. We therefore use it only for stimuli with correlation time r > 60 
msec. The slide we use shows either a natural scenery or a random vertical 
bar pattern - called "Natural" or "Bars" in the text. Although we recorded 
simultaneously from both HI neurons, for simplicity we only use one neuron 
recordings and select a subset of these for our figures. All our conclusion 
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hold for two neuron recordings [H]. 

All experiments were repeated involving about 30 flies. 

Stimulus 

The velocity stimuli v{t) were taken from Gaussian distributions with expo- 
nentially decaying correlation functions ~ exp{—t/T). 
Searching for zero crossings 

When the stimulus crosses from negative to positive values, one of the HI 
neurons typically starts to generate spikes after a delay time of ~ 40 msec. 
To locate a well deflned peak of spiking activity, we compute the spike rate - 
or the poststimulus time histogram - summing for each time bin the occuren- 
cies of spikes over all repetitions in the raster. To locate peaks, we require 
a spike rate larger than half the number of repetitions in a small window 
of a couple of msec. Prominent instants are to be preceeded by no spiking 
activity for at least 100 msec and we require peaks to be present in both 
spike trains p{t) and pnit)- Additionally no new peak is allowed for the next 
400 msec. 
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